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I. Eye Movement Control 

1. Kinetics: Time Optimal Control of Saccades 

Eye movements are elegant biological examples for application of time-optimal engineering 
control theory. The muscles that drive eyeball rotations are very long and strong and fast. The 
eyeball has low rotational inertia so that the internal viscosity of active muscle dominates plant 
dynamics; thus we may think of the system as a visco-elastic load. What controller signal shapes 
would produce a time-optimal movement? First, reciprocal innervation, proposed by Descartes 
(#149) and experimentally found by Sherrington, should be observed (Figure 1 A). Next, a set of 
multipulse-step control signals first described by Clark and Stark (#137) can be justified on theoreti- 
cal grounds by Bellman-Pontriagin bang-bang theory. 

2. Dual Mode Control: Sampled Data Operator 

Another beautiful example is the sampled data control model of Young and Stark (#27) that 
was experimentally verified by study of transient responses in normal subjects. A further exciting 
demonstration was found in the explanation of “macro-saccadic oscillation” found in patients with 
cerebellar disease that effectively removed the continual adaptive gain adjustment and left the patient 
with high gain instability oscillation (#156). (Figure IB). 

II. Higher Level EM Control and Scanpath Theory 

3. Active Looking and the Kantian Theory of Perception 

A more metaphoric theory, the Scanpath Theory has been developed by Noton and Stark 
(#1 10) and Stark and Ellis (#211) to account for the repetitive sequence of saccades that enable the 
eye and its fovea centralis to traverse over important sub-features of a picture or scene. This has been 
taken to mean than an internal cognitive model drives active looking in a top-down manner. The evi- 
dence to support this theory has to do with statistical proof that there is indeed repetition of scanpath 
sequences. Further evidence comes from studies of ambiguous figures wherein the same subject 
looking at the same picture switches both mental states of perception and patterns of repetitive sac- 
cadic sequences. Most recently, Brandt and Stark (in preparation) have used visualization tasks. A 
subject recalling a previously seen diagram demonstrates scanpath movements similar to those 
looking at the earlier viewed diagram; here only the internal model can be responsible for the scan- 
path sequence and, indeed, string editing distances support this finding quantitatively. (Figure 2A). 
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The philosopher, Kant, reasoned that internal representations (Platonic ideals) controlled active 
perception in a top-down procedure that could then organize sensation (without time and without 
space) received from the chaotic work of appearances. In this view, the virtual model within a human 
brain is more real than the external world of the naive realistic! (Figure 2B). 
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III. Top-Down Robotic Vision 


4. Models of Robots and ROIs 

For our use of visual feedback, for use by autonomous algorithms, for use by manual operators, 
and for use by supervisory controllers. We have taken this top-down scheme from human vision and 
introduced it as a basic design principle in our Telerobotic Laboratory. Instead of searching for 
important features of the TRWE (telerobotic working environment) with bottom-up image process- 
ing algorithms, we instead use a top-down approach. We already know where a particular robotic 
link is located in 3D space, we know where the cameras are pointed, we know what important 

parameters we need to measure. Thus, we use our knowledge in the form of forward-loop control 
model not only to control the robots and the cameras, but also to direct the image processing (#312 
and A304). We construct ROIs (regions-of-interest) for each camera frame so that the image 
processing can be done rapidly and robustly (Figure 3A). 

Within the ROIs, local thresholding and centroid calculations enable us to compute a feedback 
model, indicated by crosses that represent the visual measurement of the location of key on-the- 
scene- visual-enhancements . This, of course, is done dynamically so that rapid feedback permits 
closed-loop control of the robotic movement, with the human operator acting as a supervisory con- 
troller only (Figure 3B). 

IV. Over-All Robotic Control Scheme 

5. Model of RWE 

The overall model (Figure 4A) permits one to appreciate the communication functions of our 
top-down image processing scheme. Communication is restricted to parameters of the visual models, 
an enormous bandwidth reduction compared to sending video pictures of a number of views of the 
TRWE. The supervisory controller looks not at the camera view of the scene, but rather at a model of 
the TRWE! Is this going to be satisfactory? (#T41 1, #313). 

6. Evaluation of Displays and HMD 

Several evaluation studies of visual display requirements have been carried out in our labora- 
tory (#301, #A313). Dr. Won Soo Kim and colleagues have established that, although for some pur- 
poses enhanced 2D displays enable the operator to perform satisfactorily, virtual 3D displays are 
robust to various deficiencies and enable performance at least as good as the best 2D displays. Also, 
models alone or models superimposed onto video pictures are better than video pictures for short- 
term robotic control tasks (Figure 4C, #A291). In another study. Dr. Greg Tharp and colleagues 
demonstrated that 5 to 15 Hz update rates for a stereo display were fast enough to permit the subject 
to slew his head freely without suffering a penalty in performance (delays were less than 1 frame 
interval). (Figure 4B). 
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V. Summary 

Telerobotics with virtual environments is an exciting and worthwhile field of research. It offers 
challenges to engineering design of displays, of control, and of communication schemes. It relates to 
human performance since the human operator is a component either as a direct manual controller or 
as a supervisory controller. It will teach us more about human capacities and we can use knowledge 
of human plans and schemas to produce initial designs. We are also learning more about ourselves. 
Did we thing about “presence” before we considered telepresence? 


24 




Figure 1A. Descartes model. 

#149 Kenneth J. Ciuffreda and Lawrence Stark, “Descartes’ Law of Reciprocal Innervation,” American Journal of 
Optometry and Physiological Optics 52: 663-673 (1975) 
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Figure IB. SD clinical simulation. 
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SCANPATH THEORY 



Figure 2A. Scanpath theory. 
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Figure 2B. 
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Figure 3A and 3B. . 
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Figure 4A. 
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Figure 4B. 
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Figure 4C. 
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